Outline of behaviour sensitivity analysis

System dynamics models are described as being ‘causally-descriptive’, meaning that unlike other ‘black-box’ models, it is not sufficient that system dynamics only produce accurate output. In addition to being numerically valid, the behavior of system dynamics models needs to be validated.

The test presented in this document examines which input assumptions have the greatest impact on model behavior, unlike previous sensitivity analyses, which focus on numerical relationships between input assumptions and outputs (either model outcomes or policy outcomes).

Outcome Measure

The behavior of each model test was evaluated using the cumulative difference of the first derivative. The following plots explain how this outcome measure is calculated.

Figure panel A shows a plot of three outcomes: the base model, with no changes to any initial assumptions; the result varies an input assumption that produces similar model behavior; and a second test, where varying the input assumption produces different model behavior. The absolute cumulative difference between the model outcomes is plotted in panel C.

Panel B plots the first derivative of each model result. The model results with similar behavior also have a similar first derivative. Conversely, the different behavior result has differences in the derivative. The cumulative absolute difference of the first derivative shows a greater difference between the two model results, highlighting the difference between behaviors.

While the cumulative absolute difference measure cannot give insights into the model structure, it is useful to screen input assumptions that impact model behavior.

Unlike the extreme test, this analysis focuses on determining which input assumption impacts the model behavior.

Example plot for model behaviour

How to interpret the results

In this analysis, each input assumption is systematically sampled 20 times within the range of +/- 20% of the initially assumed input value. The cumulative difference of the first derivative is utilized to evaluate how each variation in the input variable influences model behavior. The average cumulative difference across the 20 samples is then summarized in the result plots.

The following figure outlines the process of generating the summary plots. Subplot A displays the average cumulative difference for the 20 samples of the metabolic equivalence of task (MET) for vigorous physical activity, presented for all tested variables.

The primary outcome, the prevalence of obesity, is depicted in subplot B for each assumption of vigorous MET value across different age groups. The subsequent derivative is illustrated in subplot C, and the cumulative difference in the derivative is shown in subplot D, with the average incremental difference highlighted in red. This average cumulative difference in the first derivative serves as the summary value presented in the summary plots.

A higher average cumulative difference in the first derivative indicates a variable with a more substantial impact on model behavior.

Overall Ranking of sensitivity

The 649 input assumptions tested have been aggregated and summarized into 70 variables by calculating the mean of the cumulative difference in derivatives for age-gender-BMI specific inputs. These 70 overarching variables are ranked in the table below.

Here is the ranking of aggregate input assumptions, where input assumptions with multiple subgroups have been averaged. For instance, changes in body weight are tested by age-gender-BMI subgroups, but the ranking is based on the mean cumulative difference in first derivatives.

The findings indicate that variables with a global impact, influencing the entire modeled population, have the most significant impact on model behavior.

Examining the most sensitive variable: energy density of sugar

Body Weight

Change in Male Body Weight input assumptions

Change in Female Body Weight input assumptions

Height

Change in Male height input assumptions

Change in Female height input assumptions

Schofield Equation - Intercept

Schofield Equation - Ceofficient

Growth Function kJ/day

Macronutrient energy density

Thermic effect of food (TEF)

Adult to Adult Social Transmission

Adult to Child Social Transmission

Child to Child Social Transmission

Infant Reported behaviours

Assumed intergenerational relationships

Mortality ratios

The Metabolic Equivalent of Task (MET)

Light physical activity

Moderate physical activity

Vigorous physical activity

Screen time

Sleep

Daily intake of Fruit

Daily intake of Vegetables

Daily intake of Grains

Daily intake of Dairy

Daily intake of Meat and Protein

Daily intake of Discretionary foods

Daily intake of Fats and Oils

Daily intake of Sugar Sweetened Beverage

Daily intake of Water

Dailyintake of Over beverages (non-water, non-ssb)

Daily intake of Other foods

Initial fat-mass (%)

Change in Male fat-mass % assumptions

Change in Female fat-mass % assumptions

Proportion of nutrients within food group Inputs

Grains

Vegetables

Fruit

Dairy

Meat and Protein

Fats

Discretionary foods

Sugar-sweetened beverages

Miscellaneous (Other)

Non-sugar-sweetened beverages

Initial BMI Prevalence Inputs

Change in Male Initial BMI Prevalence

Change in Female Initial BMI Prevalence

## R version 4.3.2 (2023-10-31)
## Platform: x86_64-pc-linux-gnu (64-bit)
## Running under: Pop!_OS 20.04 LTS
## 
## Matrix products: default
## BLAS:   /usr/lib/x86_64-linux-gnu/blas/libblas.so.3.9.0 
## LAPACK: /usr/lib/x86_64-linux-gnu/lapack/liblapack.so.3.9.0
## 
## locale:
##  [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C              
##  [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8    
##  [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8   
##  [7] LC_PAPER=en_US.UTF-8       LC_NAME=C                 
##  [9] LC_ADDRESS=C               LC_TELEPHONE=C            
## [11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       
## 
## time zone: Australia/Sydney
## tzcode source: system (glibc)
## 
## attached base packages:
## [1] stats     graphics  grDevices utils     datasets  methods   base     
## 
## other attached packages:
##  [1] kableExtra_1.3.4 scales_1.2.1     ggpubr_0.6.0     ggh4x_0.2.5     
##  [5] patchwork_1.1.3  openxlsx_4.2.5.2 lubridate_1.9.2  forcats_1.0.0   
##  [9] stringr_1.5.0    dplyr_1.1.2      purrr_1.0.2      readr_2.1.4     
## [13] tidyr_1.3.0      tibble_3.2.1     ggplot2_3.4.3    tidyverse_2.0.0 
## [17] progress_1.2.2  
## 
## loaded via a namespace (and not attached):
##  [1] gtable_0.3.4      xfun_0.40         bslib_0.5.1       rstatix_0.7.2    
##  [5] tzdb_0.4.0        vctrs_0.6.3       tools_4.3.2       generics_0.1.3   
##  [9] fansi_1.0.4       highr_0.10        pkgconfig_2.0.3   webshot_0.5.5    
## [13] lifecycle_1.0.3   farver_2.1.1      compiler_4.3.2    munsell_0.5.0    
## [17] carData_3.0-5     htmltools_0.5.6   sass_0.4.7        yaml_2.3.7       
## [21] pillar_1.9.0      car_3.1-2         crayon_1.5.2      jquerylib_0.1.4  
## [25] cachem_1.0.8      abind_1.4-5       tidyselect_1.2.0  rvest_1.0.3      
## [29] zip_2.3.0         digest_0.6.33     stringi_1.7.12    bookdown_0.35.1  
## [33] labeling_0.4.2    rmdformats_1.0.4  fastmap_1.1.1     grid_4.3.2       
## [37] colorspace_2.1-0  cli_3.6.1         magrittr_2.0.3    utf8_1.2.3       
## [41] broom_1.0.5       withr_2.5.0       prettyunits_1.1.1 backports_1.4.1  
## [45] timechange_0.2.0  rmarkdown_2.24    httr_1.4.7        ggsignif_0.6.4   
## [49] hms_1.1.3         evaluate_0.21     knitr_1.44        viridisLite_0.4.2
## [53] rlang_1.1.1       Rcpp_1.0.11       glue_1.6.2        xml2_1.3.5       
## [57] svglite_2.1.1     rstudioapi_0.15.0 jsonlite_1.8.7    R6_2.5.1         
## [61] systemfonts_1.0.4